Coflow Scheduling in Data Centers: Routing and Bandwidth Allocation

In distributed computing frameworks like MapReduce, Spark, and Dyrad, a coflow is set of flows transferring data between two stages job. The job cannot start its next stage unless all in the finish. To improve execution performance such job, it crucial to reduce completion time coflow, as can contribute more than 50 percent time. While several schedulers have been proposed, we observe that routing, factor greatly impacting Coflow Completion Time (CCT), has not well considered. this article, focus on scheduling problem jointly consider routing bandwidth allocation. We begin by providing an analytical solution optimal allocation with pre-determined routes. following, formulate single Non-linear Mixed Integer Programming present relaxed convex optimization problem. further propose algorithms, CoRBA simplified version: CoRBA-fast solve joint consideration Lastly, address multiple coflows online scheduling, scheduler named OnCoRBA. By comparing start-of-the-art algorithms via simulations, demonstrate CCT 30-400 OnCoRBA reduces average 20-230 percent. addition, be hundreds times faster around 8 degradation compared CoRBA, which makes use very appropriate practice.